STUDENT ASSESSMENT INVENTORY FOR SCHOOL DIS 


Listening to Students: 
Sample Focus Group and 
Survey Materials 


Overview 


The Student Assessment Inventory for School Districts 
supports a process to evaluate the assessments students 
currently take and then determine the minimum testing 
necessary to serve essential diagnostic, instructional and 
accountability purposes. The process should also lead to 
actions that ensure that every district-mandated assessment 
is of high quality, provides the information needed for 
specific school and district purposes, and is supported 

by structures and routines so that assessment results are 
actually used and action steps that will help students are 
taken. While the success of this process rests on several 
factors, one of the most critical is the extent to which district 
leaders intentionally incorporate the perspectives and 
experiences of students. 


Students have unique perspectives through their direct 
experiences taking tests. Their perspectives are essential for 
“taking the temperature” on the assessment environment 
in the district and building a case for action. Clearly 
communicating to students (and by extension, their 
parents) the importance and urgency of this effort will 

help ensure momentum leading to action. Additionally, 
students’ experiences with assessments are foundational 

to understanding the assessments themselves, how the 
assessment results are being communicated and understood, 
and what can be done to make individual assessments and 
the district’s assessment program as a whole more effective 
and coherent. 


CC BY 4.0 Achieve 2015. Districts may use or adapt. If modified, please attribute Achieve and re-title. 


This resource is designed to guide districts in the steps 
necessary to design and use focus groups and surveys 
to incorporate student perspectives, experiences and 
expertise in the inventory process. It includes two types of 


information. 


e Instructions. Achieve recommends following a set 
of steps to design, build, use and analyze student 
surveys and focus groups to effectively incorporate this 
information in the inventory process and build toward 
actionable recommendations. A set of guidance and 
resources (beginning on page 2) addresses common 
questions about how and when to use student focus 
groups and surveys within the inventory process. It also 
includes design considerations district leaders should keep 
in mind as they develop these feedback instruments, use 
information gathered from students, and communicate 
findings to participants and stakeholders. Districts 
experienced in designing focus groups and surveys may 
feel comfortable skipping this section and move directly 


to the protocols, beginning on page 6. 


e Inventory Tools. This resource provides student 
focus group and survey questions that district leaders 
can use or adapt as part of their local assessment 
inventory process. Taking the Temperature on 
Assessments (beginning on page 6) focuses on 
illuminating student perspectives on the district’s 
assessment program as a whole to build a greater 
understanding of the testing environment and Z 
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help build a case for action. Depending on district 
context, leadership teams may decide to use focus 
groups or surveys to illuminate student perspectives, 
experiences and expertise. As such, this guide includes 


questions appropriate for each of these formats. 


Instructions: 
Guidance and Resources 


This guidance is intended to assist district leaders in 
developing a process for using student focus groups and/or 
surveys as part of the assessment inventory process. 


1. DETERMINING DISTRICT NEEDS FOR 
STUDENT FEEDBACK BASED ON THE 
REFLECT AND PLAN QUESTIONS IN THE 
STUDENT ASSESSMENT INVENTORY FOR 
SCHOOL DISTRICTS 


At the outset of the assessment inventory process, district 
leaders should address the Reflect and Plan questions on 
page 4 of the Student Assessment Inventory for School 
Districts. These questions ask districts to think about the 
context, objectives and structure of the inventory process. 
District leaders are then encouraged to develop a strong 
understanding of student perceptions of testing through the 
Taking the Temperature on Assessments focus group and 


survey protocols beginning on page 6. 


2. DECIDING WHETHER TO USE A FOCUS 
GROUP, SURVEY OR BOTH 


Districts have multiple issues to consider when determining 
whether to use focus groups or surveys to gauge student 


perceptions. Focus groups allow for additional probing 


and follow-up questions. On the other hand, focus groups 
are generally more costly than surveys to organize, staff 

and analyze, and they require access to skilled focus group 
moderators. Surveys allow districts to gather information 
from a large number of students in a relatively short period 
of time, and answers can be easily aggregated. Students 

may also feel more comfortable completing a survey than 
talking in a focus group. However, surveys do not allow for 
additional probing questions, and response rates may be low, 
especially if a survey takes too long to complete. Districts 

are encouraged to carefully weigh these issues in light of 
resource and timeline considerations, as well as local context 


and needs. 


3. DESIGNING A SUCCESSFUL FOCUS GROUP 
OR SURVEY PROCESS 


There are multiple design considerations for focus groups 
and surveys. These considerations include focus group and 
survey administration, participation, setting and design. The 
focus group and survey protocols included in this resource 
are openly licensed, and districts are encouraged to add and/ 
or adapt questions and protocols as needed based on local 
priorities and context. Below, focus group and survey design 


considerations are described separately. 


Focus Group Design Considerations 


Who should administer a focus group? District leaders can 
choose to administer their own focus groups or work with 
an outside organization with experience conducting focus 
groups. If district leaders decide to keep focus groups in 
house, it is important that focus group moderators are 
comfortable speaking with students, have a high level 

of rapport with participants, and are people with whom 


students are likely to feel they can be open and honest. If 
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district leaders have limited time or capacity for conducting 
focus groups, they can consider using the expertise of 

local institutes of higher education, community-based 
organizations focused on education and consultants with 


focus group expertise. 


Who should participate in a focus group? District leaders 
should make every effort to recruit students with diverse 
perspectives and school experiences for focus groups. 
Districts should strive to select a representative sample of 
students across multiple criteria (e.g., grade level or span; 
school; or student subgroup, such as English language 
learner or special education). District leaders should also 
ensure that focus groups are demographically representative 
of the community, taking into consideration characteristics 
such as race/ethnicity, gender, neighborhood, etc. Districts 
should also consider how best to gather meaningful data 
from different age groups. Questions may need to be 
significantly different for high school, middle school and 


elementary school students. 


Additionally, district leaders should carefully consider 

the number of focus groups that will be needed to get a 
detailed picture of a district’s assessment landscape, as 

well as the number of participants that can be reasonably 
accommodated in any one focus group. To allow all 
participants multiple opportunities to contribute to the 
conversation, it is suggested that focus groups not exceed 

10 individuals. It is also recommended that the focus group 
moderator be accompanied by a dedicated note-taker so that 


all responses can be recorded. 


What are the best opportunities to conduct a focus group? 
Districts or third-party organizations can use a range of 
formal and informal opportunities to gather feedback 
through focus groups. There may be already-scheduled 
events where large numbers of students will be present, 
such as back-to-school nights, which can be convenient and 
minimally disruptive for participants. Gathering informal 
feedback through small ad-hoc focus groups is another key 
opportunity, especially for districts with limited resources or 
shorter timelines for the inventory process. 


How can districts ensure that student focus groups are 
diverse and representative of the student population? 
Reaching a representative sample of students for focus 
groups presents unique challenges for districts. Districts 

can develop a set of strategies prior to engaging in focus 
groups. First, districts are encouraged to partner with 
community-based youth organizations to reach out to 
students from underrepresented groups. Second, when 
developing focus group participation lists, districts should 
compare the demographics of student participants to district 
enrollment information to ensure that a representative group 
of students has been reached when initial rounds of focus 
groups are completed. Districts should continue to monitor 
participant demographics throughout the process to strive 
for representativeness. Finally, districts should consider 
translating focus group protocols into additional languages 
spoken by students and having translators lead or assist with 


student focus groups where needed. 


What additional safeguards should be in place when 
gathering information from students? Districts or external 
organizations should be aware of federal, state and district 
policies that protect students’ right to privacy. Districts may 
want to consider guaranteeing anonymity to focus group and 


survey participants. 
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Survey Design Considerations 


How should surveys be used to generate the most useful 
information? District leaders should address several 
considerations as they design surveys as part of the inventory 


process. 


e Choose only the questions that are most likely to 


generate the information needed. 


e In prioritizing some questions over others, be sensitive 
to the time needed to complete the survey. 


e Choose question types that best match needs and 
capacity to analyze data. A variety of question 
types are included in these sample surveys, all with 
different implications for data analysis. For instance, 
open-ended questions have the benefit of enabling 
respondents to provide their personal feedback, but 
they can be time consuming to analyze when looking 
at hundreds of responses. Conversely, multiple-choice 
responses are very easy to code for data analysis but 
do not typically give respondents an opportunity to 
provide in-depth feedback. 


e After identifying the questions that would be most 
helpful, edit them as needed to ensure that they fit 
the local context and will resonate with respondents. 
Phrases in brackets within the template indicate 
places where the language needs to be customized or 
removed prior to being released publicly. These areas 
should be updated with the appropriate information. 


e Decide whether to ask respondents to provide the 
name of their school or other identifying information 
as part of this survey. Districts and external 
organizations should follow federal, state and local 
laws safeguarding students’ privacy. 


e Prior to implementing the survey, try out the 
survey items on a small number of potential survey 
participants to get their feedback on the questions 
asked, clarity of questions and length of time the 


survey took. 


Who should administer a survey? Surveys can be 
administered by a district or third-party organization. District 
leaders may want to administer their own surveys if they 
have prior experience developing and administering them, 
have had good response rates with previous surveys, and/or 
have sufficient capacity and expertise. District leaders may 
want to consider using an external, third-party organization 
to administer a survey if sufficient resources exist to hire an 
outside firm with expertise, the district has little experience 
designing and administering its own surveys, and/or privacy 


and anonymity of results are significant concerns. 


Who should participate in a survey? Depending on district 
size, resources and capacity, district leaders can choose 

to either survey all students (census survey) or select a 
representative sample of students across multiple criteria 
(e.g., grade level or span; school; or student subgroup, such 
as English language learner or special education). Districts 
should also strive to collect surveys from a demographically 
representative sample of students, taking into 
consideration characteristics such as race/ethnicity, gender, 
neighborhood, etc. (see below). If initial survey results are 
not representative of the district’s student demographics, 
districts should consider targeting additional individuals from 


underrepresented groups with follow-up survey requests. 


How can districts ensure that survey responses are from a 
diverse and representative sample of students? Similar to 
focus groups, reaching a representative sample of students 
through a survey presents unique challenges for districts. 
First, districts should consider translating student surveys 
into additional languages spoken by students. Second, if the 
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survey is paper-based, districts can use a range of formal 
and informal opportunities to gather survey responses, 

in much the same way as focus groups described above. 

For example, homeroom/advisory classes are potential 
opportunities to have students complete surveys. For online 
surveys, districts should be aware that not all students have 
internet access and provide multiple opportunities — for 
example, at schools and libraries — for students to complete 
surveys. Third, districts may want to consider partnering 
with community-based youth organizations to reach out to 
students from underrepresented groups. Finally, when initial 
rounds of surveys are completed, districts should compare 
the demographics of student participants to overall student 
demographics to ensure that a representative group of 


students has been reached. 


4. NEXT STEPS: ANALYSIS, USE AND 
COMMUNICATION OF RESULTS 


After districts have conducted all student focus groups and/ 
or surveys, information gathered from these sources can 
be analyzed and incorporated into the Inventory Table (see 


pages 8 and 9 of the Student Assessment Inventory for School 


Districts). \t is recommended that districts also communicate 


results of the focus groups and surveys to participants (and 
depending on timing and context, the public) to demonstrate 


that students’ voices are being heard in the inventory process. 


Analyzing and using results. Focus groups and surveys 

are part of the inventory process and will help inform the 
Inventory Table, overall analysis and recommendations. 
Please refer to pages 5—6 of the Student Assessment 
Inventory for School Districts. Analyzing the responses may 
raise additional questions about student perceptions and 
experiences and the impact of assessments on students. For 
example, do particular assessments mentioned by students 


fall within a specific grade level or grade band, or are these 
assessments taken by a specific subgroup of students? Are 
there concerns about the amount of testing in particular 
grades or subjects? This information from students may also 
help inform the district about communicating the results of 
particular tests to the broader community, as well as inform 


systemic needs around assessment and data literacy. 


Communicating the results to participants and the public. \t 
is important that district leaders communicate to focus group 
and survey participants that their responses and suggestions 
are being heard. Examples of communication to participants 
could include a letter to participants with a summary of what 
was learned through the sessions or survey, an invitation 

to a district or board meeting during which results will be 
discussed, and a timeline for moving from feedback to 
recommendations to action. District leaders may also want 
to consider releasing a summarized set of findings from the 
focus groups and surveys to the broader public as an interim 
step during the inventory process. 


Additional Resources 


These additional resources may be helpful in designing and 


administering surveys and focus groups. 


Achieve (2012). Feedback Loops for Common Core Standards 
Implementation: Survey Tools for State Education Leaders. 
www.achieve.org/files/GuidanceforsurveysFINAL6-25-12- 
TOSHAREv2. pdf 


Duke Initiative on Survey Methodology. 


http://dism.ssri.duke.edu/resources.php 


Pew Research. Questionnaire Design. 


www.people-press.org/methodology/questionnaire-design 
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Taking the Temperature on Assessments 


context. 


This section, which includes sample focus group and survey questions, focuses on 
illuminating students’ perspectives on the district’s assessment program as a whole to 
build a greater understanding of the testing environment. It is recommended that these 
questions be asked of students toward the beginning of the process, prior to the Inventory 
Table being completed. Conducting focus groups or administering surveys early in the 
process helps make the case for why an inventory process is needed and will lead to 
concrete action to ensure that every assessment being used is useful and of high quality. 


The protocols and questions listed below are openly licensed, and districts are encouraged 
to add and/or adapt questions and protocols as needed based on local priorities and 


TAKING THE TEMPERATURE ON ASSESSMENTS: 
SAMPLE FOCUS GROUP QUESTIONS 


INTRODUCTION AND WARM-UP 


A warm-up period for focus groups is important to help 
participants understand the goals for the session, create 

a common understanding around how participants and 

the moderator will interact, and build comfort among 
participants and with the moderator. Below are a suggested 
set of “ground rules” and an introductory exercise, followed 


by sample focus group questions. 


Note to moderator: Prior to conducting this focus group, it 
is important to be clear with participants about the types 
of assessments for which the district would like feedback. 
Will there be discussion only of assessments that are given 
across multiple classrooms or schools as part of a district/ 


school assessment program, or is the district also interested 


in teacher-developed classroom assessments? This should be 
discussed as part of the introduction and warm-up. Add and/ 
or adapt questions to best meet your local needs and context. 


e Introduction of moderator and focus group 
guidelines: 


> We’re going to be talking about testing in your school. 
This will help the district better understand the 


volume, quality and use of tests given in the district. 


> The district will look at all of its testing through 
the perspective of students. It is important to 
understand how different kinds of students 
experience testing. That is why we have invited you 


to participate in these focus groups. 
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> This focus group is part of a larger process to 
examine many kinds of tests given to students. 
Through this process, the district will make decisions 
about which tests are high quality and necessary for 
students to take and which tests can be modified or 
eliminated. 


> One person should speak at a time. 


> There are no “wrong” answers to any of these 
questions. We are interested in hearing your 
perspectives as students. 


> We value confidentiality. We will not associate any 
feedback that comes out of this focus group with a 
particular individual. 


> Everyone will have a chance to speak. If you have 
not had an opportunity to provide your perspective, 
| may call on you. 


> Please turn off or silence your cell phones. 


> Are there additional guidelines the group would like 
to add? 


Disclosure of note-taking: 


> Anote-taker is present to make sure that we get all 
of your feedback. The note-taker will not associate 


comments with names. 


e Parking lot: 


> There is a “parking lot” chart/board/area where 
we can put ideas or thoughts that come up in 
our discussion that are important but may not be 
related to the purpose of this group. We want to 
capture those important thoughts, but we also want 
to keep focused on the purpose of our meeting. 
These ideas or thoughts will be shared with 


appropriate individuals following the meeting. 


e Communicating results of focus groups: 


> The results of the focus groups will be summarized, 
and you will receive a summary by {date}. If we 
missed any key points you raised during this 


conversation, please let us know. 
Introductory exercise (possible options): 
> Types of tests given to students 
> Examples of tests given in the district 


> What questions do you have about testing before 
we begin? 


Introductory exercise (possible options): 
> How long have you lived in this community? 


> What kinds of activities are you involved in outside 


of school? 


> What do you like to do for fun? 
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FOCUS GROUP QUESTIONS 


1. 


What grade are you in? Which school do you 
attend? 


What kinds of tests have you taken in the last year? 
(Moderator prompt: tests designed by your teacher 
for just your class [e.g., chapter tests, mid-terms, 
finals], tests all students in your school take [e.g., 
schoolwide writing prompts, district benchmark 
tests, statewide tests {e.g., PARCC, Smarter 
Balanced}J, tests you take for college entrance or 
honors [e.g., AP, IB, ACT, SAT]) 


Do you think students in this district are given too 
many tests, not enough tests or about the right 
number of tests throughout the school year? Why 
do you say that? 


What are some examples of tests that you have 
found useful in understanding what you’ve learned 
and need to do next? What has made them useful? 
(Moderator prompt: you get the results quickly and 
they are clear, test measures important material 
covered in class, test helps measure readiness for 
college or career) 


What are some examples of tests that you have not 
found useful in understanding what you’ve learned 
and what you need to do next? Why have they not 
been useful? (Moderator prompt: you don’t get 

the results for a long time or at all, test measures 
material not covered in your class, is not helpful in 


measuring readiness for college or career) 


How do you think your teachers use the results 
from the tests you take? How do they talk to you 
about test results? How do you talk about them 
with your parents/family? 


What tests, if any, do you think you should keep 
taking? Why do you say that? 


What tests, if any, do you think you should not 
keep taking or should be changed? Why do you say 
that? (Moderator prompt: you do not have enough 
time to take the test, too much time is spent taking 
the test, tests should include different kinds of 
questions) 


Do you have any other suggestions on how to 
improve the tests you take? 
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TAKING THE TEMPERATURE ON ASSESSMENTS: 
SAMPLE SURVEY QUESTIONS 


Sample language to introduce the survey: This survey asks 
questions about tests that you take. This survey will help 
your school and your district make good decisions about 
testing. Your responses to this survey are confidential — we 
do not ask for your name and will not ask for any personal 
information. If you have questions about this survey, please 
contact{ }. 


What grade are you in? 
{district should prepopulate with grades} 


Do you think students in your school are given too 
many tests, not enough tests or about the right number 
of tests throughout the school year? 


1 Not enough tests 


(1 About the right number of tests 


1 Too many tests 


What concerns or worries do you have about the tests 
you take? {open-ended item} 


Note to survey designer: Prior to administering this survey, 

it is important to be clear with respondents about the types 
of assessments for which the district would like feedback. 

Is the primary focus on assessments given across multiple 
classrooms or schools as part of a district/school assessment 
program, or is the district also interested in teacher- 
developed classroom assessments? The district should also 
explain how results of the survey will be used, whether 
district stakeholders will be presented with a summary of 


findings and how these findings will be used to inform action. 


4. What are some examples of tests that you have found 
helpful in understanding what you’re learning and what 
you need to do next? {the district can prepopulate this item 


with a list of assessments given in the district if known} 


5. In general, what about these tests make them helpful? 


Connected to what | am learning in the classroom 
Results are available soon after taking the test 


Scores and information provided to me are easy to 
understand 


Provides ideas on how to help me improve in my classes 


1 Results show me how my scores compare to the 
scores of other students in my school, district and/ 


or state 


J Other 
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What are some examples of tests that you have not 


found helpful in understanding what you’re learning 


and what you need to do next? {the district can 


prepopulate this item with a list of assessments given 


in the district if known} 


In general, what about these tests makes them not 
helpful? 


= 


| do not understand the connection to material | am 


learning in class 
| do not receive results quickly 


Scores and information provided to me are difficult 
to understand 


Does not give me ideas on how to help me improve 
in my classes 


Other 


10. 


11. 


12. 


How do you think your teachers use the results from 
the tests you take? How do they communicate with you 
about test results? {open-ended item} 


How do you talk with your parents/family about test 
results? {open-ended item} 


What tests, if any, do you think you should keep 
taking? {open-ended item OR district can list specific 
assessments} 


What tests, if any, do you think you should not keep 
taking? {open-ended item OR district can list specific 


assessments} 


Are there any other suggestions you want to make 
about tests that you take? {open-ended item} 
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